{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PaddlePaddle实现线性回归-房价预测模型\n",
    "\n",
    "## 基本概念\n",
    "\n",
    "线性回归是机器学习中最简单也是最重要的模型之一，其模型建立同样遵循上图流程：获取数据、数据预处理、训练模型、应用模型。回归模型可以理解为：存在一个点集，用一条曲线去拟合它分布的过程。如果拟合曲线是一条直线，则称为线性回归。如果是一条二次曲线，则被称为二次回归。线性回归是回归模型中最简单的一种。读者可以看到图中成本在刚开始收敛较快，随着迭代次数变多，收敛速度变慢，最终收敛到一个较小值。\n",
    "\n",
    "\n",
    "在线性回归中有几个基本的概念需要掌握：\n",
    "\n",
    "- 假设函数（Hypothesis Function）：用数学的方法描述自变量和因变量之间的关系，它们之间可以是一个线性函数或非线性函数\n",
    "\n",
    "- 损失函数（Loss Function）：用数学的方法衡量假设函数预测结果与真实值之间的误差。这个差距越小预测越准确，而算法的任务就是使这个差距越来越小。\n",
    "\n",
    "- 优化算法（Optimization Algorithm)：它决定了一个模型的精度和运算速度。本章的线性回归实例中主要使用了梯度下降法进行优化。\n",
    "\n",
    "** 损失函数 ** \n",
    "\n",
    "对于某个具体样本(x(i),y(i))，算法通过不断调整参数值和，最终使得预测值和真实值尽可能相似，即 。整个训练的过程可以表述为通过调整参数值和最小化损失函数。因此，损失函数也是衡量算法优良性的方法。这里涉及到两个值：预测值和真实值。预测值是算法给出的值（用来表示概率）。而真实值是训练集中预先包含的，是事先准备好的。形式上，可以表示为：\n",
    "\n",
    "$$\\{(x^{(1)}, y^{(1)}), (x^{(2)}, y^{(2)}), ... ,(x^{(m)}, y^{(m)})\\}$$\n",
    "\n",
    "其中，x(i)表示属于第i个样本的特征向量，y(i)表示属于第i个样本的分类标签，也就是真实值。损失函数的选择需要具体问题具体分析，在不同问题场景下采用不同的函数。通常情况下，会将损失函数定义为平方损失函数（Quadratic Loss Function）。在本次线性回归中，使用的是均方差（Mean Squared Error）来衡量。\n",
    "$$L(\\hat{y} , y) = -(ylog{ \\hat{y} } + (1-y)log(1 - \\hat{y}))$$\n",
    "\n",
    "** 梯度下降算法 **\n",
    "\n",
    "梯度下降是深度学习中非常重要的概念，值得庆幸的是它也十分容易理解。\n",
    "\n",
    "<img src=\"image/gd.png\" style=\"height:240px;width:500px;\">\n",
    "\n",
    "损失函数J(w,b)可以理解为变量w和b的函数。观察图，垂直轴表示损失函数的值，两个水平轴分别表示变量w和b。实际上，w可能是更高维的向量，但是为了方便说明，在这里假设w和b都是一个实数。算法的最终目标是找到损失函数J(w,b)的最小值。而这个寻找过程就是不断地微调变量w和b的值，一步一步地的试出这个最小值。而试的方法就是沿着梯度方向逐步移动。本例中让图中的圆点表示J(w,b)的某个值，那么梯度下降就是让圆点沿着曲面下降，直到J(w,b)取到最小值或逼近最小值。\n",
    "\n",
    "了解了以上几个关键概念后，我们以预测房价为背景，介绍线性回归的PaddlePaddle实现过程。数据集使用了某地区的房价分布，为了简化模型数据只有两维，分别是房屋面积与房屋价格。可以看到房价与房屋面积之间存在一种关系，这种关系究竟是什么，就是本次预测想要得到的结论。\n",
    "<img src=\"image/house.png\" style=\"height:150px;width:120px;\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1 - 引用库\n",
    "\n",
    "在进行网络配置之前，首先需要加载相应的Python库，并进行初始化操作。\n",
    "- numpy：一个python的基本库，用于科学计算\n",
    "- matplotlib.pyplot：用于生成图，在验证模型准确率和展示成本变化趋势时会使用到\n",
    "- paddle.v2：paddle深度学习平台"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import paddle.v2 as paddle\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2 - 数据预处理\n",
    "\n",
    "本次数据集使用的是2016年12月份某市某地区的房价分布。为了简化模型，假设影响房价的因素只有房屋面积，因此数据集只有两列。下述代码展示了数据处理的全部过程，包括数据装载和归一化。\n",
    "\n",
    "** 初始化全局变量 **\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 训练数据、原始数据、测试数据\n",
    "CODEMASTER_TRAIN_DATA = None\n",
    "X_RAW = None\n",
    "CODEMASTER_TEST_DATA = None\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** 载入数据并进行预处理 **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 载入数据\n",
    "def load_data(filename, feature_num=2, ratio=0.8):\n",
    "    \"\"\"\n",
    "    载入数据并进行数据预处理\n",
    "\n",
    "    Args:\n",
    "        filename -- 数据存储文件，从该文件读取数据\n",
    "        feature_num -- 数据特征数量\n",
    "        ratio -- 训练集占总数据集比例\n",
    "    Return:\n",
    "    \"\"\"\n",
    "    # 如果测试数据集和训练数据集都不为空，就不再载入数据load_data\n",
    "    global CODEMASTER_TRAIN_DATA, CODEMASTER_TEST_DATA, X_RAW\n",
    "    if CODEMASTER_TRAIN_DATA is not None and CODEMASTER_TEST_DATA is not None:\n",
    "        return\n",
    "    # data = np.loadtxt()表示将数据载入后以矩阵或向量的形式存储在data中\n",
    "    # delimiter=',' 表示以','为分隔符\n",
    "    data = np.loadtxt(filename, delimiter=',')\n",
    "    X_RAW = data.T[0].copy()\n",
    "    # axis=0 表示按列计算\n",
    "    # data.shape[0]表示data中一共多少列\n",
    "    maximums, minimums, avgs = data.max(axis=0), data.min(axis=0), data.sum(\n",
    "        axis=0) / data.shape[0]\n",
    "\n",
    "    # 归一化，data[:, i] 表示第i列的元素\n",
    "    for i in xrange(feature_num - 1):\n",
    "        data[:, i] = (data[:, i] - avgs[i]) / (maximums[i] - minimums[i])\n",
    "\n",
    "    # offset用于划分训练数据集和测试数据集，例如0.8表示训练集占80%\n",
    "    offset = int(data.shape[0] * ratio)\n",
    "    CODEMASTER_TRAIN_DATA = data[:offset].copy()\n",
    "    CODEMASTER_TEST_DATA = data[offset:].copy()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "完成数据集的加载和分割后，PaddlePaddle中通过reader来加载数据。 Reader返回的数据可以包括多列，利用reader可以使训练组合特征变得更容易。本次的数据只有两维，reader优点不明显，当数据多起来时就会发现利用多个reader训练模型会比传统方法方便许多。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 读取训练数据或测试数据，服务于train()和test()\n",
    "def read_data(data_set):\n",
    "    \"\"\"\n",
    "        一个reader\n",
    "        Args:\n",
    "            data_set -- 要获取的数据集\n",
    "        Return:\n",
    "            reader -- 用于获取训练数据集及其标签的生成器generator\n",
    "    \"\"\"\n",
    "    def reader():\n",
    "        \"\"\"\n",
    "        一个reader\n",
    "        Args:\n",
    "        Return:\n",
    "            data[:-1], data[-1:] -- 使用yield返回生成器(generator)，\n",
    "                    data[:-1]表示前n-1个元素，也就是训练数据，data[-1:]表示最后一个元素，也就是对应的标签\n",
    "        \"\"\"\n",
    "        for data in data_set:\n",
    "            yield data[:-1], data[-1:]\n",
    "    return reader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 获取训练数据集\n",
    "def train():\n",
    "    \"\"\"\n",
    "    定义一个reader来获取训练数据集及其标签\n",
    "\n",
    "    Args:\n",
    "    Return:\n",
    "        read_data -- 用于获取训练数据集及其标签的reader\n",
    "    \"\"\"\n",
    "    global CODEMASTER_TRAIN_DATA\n",
    "    load_data('data.txt')\n",
    "\n",
    "    return read_data(CODEMASTER_TRAIN_DATA)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 获取测试数据集\n",
    "def test():\n",
    "    \"\"\"\n",
    "    定义一个reader来获取测试数据集及其标签\n",
    "\n",
    "    Args:\n",
    "    Return:\n",
    "        read_data -- 用于获取测试数据集及其标签的reader\n",
    "    \"\"\"\n",
    "    global CODEMASTER_TEST_DATA\n",
    "    load_data('data.txt')\n",
    "\n",
    "    return read_data(CODEMASTER_TEST_DATA)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3 - 配置网络结构\n",
    "\n",
    "线性回归的模型其实就是一个采用线性激活函数（linear activation，LinearActivation）的全连接层（fully-connected layer，fc_layer），因此在Peddlepeddle中利用全连接层模型构造线性回归，这样一个全连接层就可以看做是一个简单的神经网络。本次的模型由于只有一个影响参数，因此输入只含一个。\n",
    "\n",
    "<img src=\"image/linear.png\" style=\"width:400px;height:250px;\">\n",
    "\n",
    "搭建神经网络就像使用积木搭建宝塔一样。在PaddlePaddle中，网络层（layer）是我们的积木，而神经网络是我们要搭建的宝塔。我们使用不同的layer进行组合，来搭建神经网络。 宝塔的底端需要坚实的基座来支撑，同样，神经网络也需要一些特定的layer作为输入接口，来完成网络的训练。\n",
    "\n",
    "下述代码定义了一个layer组合，其中，x与y为之前描述的输入层；y_predict接收x作为输入，接上一个全连接层；cost接收y_predict与y作为输入，接上均方误差层。最后一层cost中记录了神经网络的所有拓扑结构，通过组合不同的layer，我们即可完成神经网络的搭建。其中x表示输入数据是一个维度为1的稠密向量，y表示输入数据是一个维度为1的稠密向量。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "#配置网络结构\n",
    "def netconfig():\n",
    "    \"\"\"\n",
    "    配置网络结构\n",
    "    Args:\n",
    "    Return:\n",
    "        image -- 输入层，DATADIM维稠密向量\n",
    "        y_predict -- 输出层，Linear作为激活函数\n",
    "        y_label -- 标签数据，1维稠密向量\n",
    "        cost -- 损失函数\n",
    "        parameters -- 模型参数\n",
    "        optimizer -- 优化器\n",
    "        feeding -- 数据映射，python字典\n",
    "    \"\"\"\n",
    "    # 输入层，paddle.layer.data表示数据层,name=’x’：名称为x,\n",
    "    # type=paddle.data_type.dense_vector(1)：数据类型为1维稠密向量\n",
    "    x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(1))\n",
    "\n",
    "    # 输出层，paddle.layer.fc表示全连接层，input=x: 该层输入数据为x\n",
    "    # size=1：神经元个数，act=paddle.activation.Linear()：激活函数为Linear()\n",
    "    y_predict = paddle.layer.fc(input=x, size=1, act=paddle.activation.Linear())\n",
    "\n",
    "    # 标签数据，paddle.layer.data表示数据层，name=’y’：名称为y\n",
    "    # type=paddle.data_type.dense_vector(1)：数据类型为1维稠密向量\n",
    "    y = paddle.layer.data(name='y', type=paddle.data_type.dense_vector(1))\n",
    "\n",
    "    # 定义成本函数为均方差损失函数square_error_cost\n",
    "    cost = paddle.layer.square_error_cost(input=y_predict, label=y)\n",
    "\n",
    "    # 利用cost创建parameters\n",
    "    parameters = paddle.parameters.create(cost)\n",
    "\n",
    "    # 创建optimizer，并初始化momentum\n",
    "    optimizer = paddle.optimizer.Momentum(momentum=0)\n",
    "\n",
    "    # 数据层和数组索引映射，用于trainer训练时喂数据\n",
    "    feeding = {'x': 0, 'y': 1}\n",
    "\n",
    "    data = [x, y_predict, y, cost, parameters, optimizer, feeding]\n",
    "\n",
    "    return data\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4 - 训练过程\n",
    "\n",
    "在完成神经网络结构配置之后，首先需要根据神经网络结构来创建所需要优化的参数集合（parameters），并创建优化器（optimizer）。之后，接下来可以创建训练器（trainer）来对网络进行训练。其中，trainer接收三个参数， 配置三个参数cost、parameters、update_equation，它们分别表示成本函数、参数和更新公式。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO 2017-12-25 06:32:39,183 networks.py:1482] The input order is [x, y]\n",
      "[INFO 2017-12-25 06:32:39,190 networks.py:1488] The output order is [__mse_cost_0__]\n",
      "[INFO 2017-12-25 06:32:39,197 networks.py:1482] The input order is [x, y]\n",
      "[INFO 2017-12-25 06:32:39,202 networks.py:1488] The output order is [__mse_cost_0__]\n"
     ]
    }
   ],
   "source": [
    "# 初始化，设置是否使用gpu，trainer数量\n",
    "paddle.init(use_gpu=False, trainer_count=1)\n",
    "\n",
    "x, y_predict, y, cost, parameters, optimizer, feeding = netconfig()\n",
    "\n",
    "# 记录成本cost\n",
    "costs = []\n",
    "\n",
    "# 创建trainer\n",
    "trainer = paddle.trainer.SGD(cost=cost, parameters=parameters, update_equation=optimizer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在配置神经网络结构的过程中，仅仅对神经网络的输入进行了描述。而trainer需要读取训练数据进行训练，PaddlePaddle中通过reader来加载数据。 Reader返回的数据可以包括多列，一个Python dict可以把列序号映射到网络里的数据层。此外，PaddlePaddle还提供事件管理机制event handler，可以用来打印训练的进度及对模型训练效果进行监控。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 打印训练过程信息\n",
    "def event_handler(event):\n",
    "    \"\"\"\n",
    "    事件处理器，可以根据训练过程的信息作相应操作\n",
    "    Args:\n",
    "        event -- 事件对象，包含event.pass_id, event.batch_id, event.cost等信息\n",
    "    Return:\n",
    "    \"\"\"\n",
    "    if isinstance(event, paddle.event.EndIteration):\n",
    "        if event.pass_id % 100 == 0:\n",
    "            print \"Pass %d, Batch %d, Cost %f\" % (\n",
    "                event.pass_id, event.batch_id, event.cost)\n",
    "            costs.append(event.cost)\n",
    "\n",
    "    if isinstance(event, paddle.event.EndPass):\n",
    "        result = trainer.test(\n",
    "            reader=paddle.batch(test(), batch_size=2),\n",
    "            feeding=feeding)\n",
    "        print \"Test %d, Cost %f\" % (event.pass_id, result.cost)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** 模型训练 **\n",
    "\n",
    "上述内容进行了初始化并配置了网络结构，接下来利用上述配置进行模型训练。\n",
    "\n",
    "首先定义一个随机梯度下降trainer，配置三个参数cost、parameters、update_equation，它们分别表示成本函数、参数和更新公式。\n",
    "\n",
    "再利用trainer.train()即可开始真正的模型训练：\n",
    "- paddle.reader.shuffle(train(), buf_size=500)表示trainer从train()这个reader中读取了buf_size=500大小的数据并打乱顺序\n",
    "- paddle.batch(reader(), batch_size=256)表示从打乱的数据中再取出batch_size=256大小的数据进行一次迭代训练\n",
    "- 参数feeding用到了之前定义的feeding索引，将数据层image和label输入trainer，也就是训练数据的来源。\n",
    "- 参数event_handler是事件管理机制，读者可以自定义event_handler，根据事件信息作相应的操作。\n",
    "- 参数num_passes=300表示迭代训练300次后停止训练。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pass 0, Batch 0, Cost 451396.781250\n",
      "Pass 0, Batch 1, Cost 161123.718750\n",
      "Pass 0, Batch 2, Cost 95150.760870\n",
      "Test 0, Cost 78716.032743\n",
      "Test 1, Cost 64055.805154\n",
      "Test 2, Cost 58688.981138\n",
      "Test 3, Cost 54197.818933\n",
      "Test 4, Cost 50216.046274\n",
      "Test 5, Cost 46687.523248\n",
      "Test 6, Cost 43556.274846\n",
      "Test 7, Cost 40789.440595\n",
      "Test 8, Cost 38342.586017\n",
      "Test 9, Cost 36154.617022\n",
      "Test 10, Cost 34250.378032\n",
      "Test 11, Cost 32576.608462\n",
      "Test 12, Cost 31059.964123\n",
      "Test 13, Cost 29719.393458\n",
      "Test 14, Cost 28565.036137\n",
      "Test 15, Cost 27484.750906\n",
      "Test 16, Cost 26589.802841\n",
      "Test 17, Cost 25768.127287\n",
      "Test 18, Cost 25128.457994\n",
      "Test 19, Cost 24412.014256\n",
      "Test 20, Cost 23834.376012\n",
      "Test 21, Cost 23442.690983\n",
      "Test 22, Cost 22993.790680\n",
      "Test 23, Cost 22570.143308\n",
      "Test 24, Cost 22251.150047\n",
      "Test 25, Cost 21959.284203\n",
      "Test 26, Cost 21634.785514\n",
      "Test 27, Cost 21432.838187\n",
      "Test 28, Cost 21252.535045\n",
      "Test 29, Cost 21123.863543\n",
      "Test 30, Cost 20935.006401\n",
      "Test 31, Cost 20836.804354\n",
      "Test 32, Cost 20737.280186\n",
      "Test 33, Cost 20664.526045\n",
      "Test 34, Cost 20597.328805\n",
      "Test 35, Cost 20473.981434\n",
      "Test 36, Cost 20379.205978\n",
      "Test 37, Cost 20304.314638\n",
      "Test 38, Cost 20318.084356\n",
      "Test 39, Cost 20253.258144\n",
      "Test 40, Cost 20196.682658\n",
      "Test 41, Cost 20105.788573\n",
      "Test 42, Cost 20118.195416\n",
      "Test 43, Cost 20056.974410\n",
      "Test 44, Cost 20089.433891\n",
      "Test 45, Cost 20016.743382\n",
      "Test 46, Cost 20012.693651\n",
      "Test 47, Cost 20042.380577\n",
      "Test 48, Cost 20054.204211\n",
      "Test 49, Cost 20013.241552\n",
      "Test 50, Cost 20063.911524\n",
      "Test 51, Cost 20105.909448\n",
      "Test 52, Cost 20068.752159\n",
      "Test 53, Cost 19963.863227\n",
      "Test 54, Cost 19978.359476\n",
      "Test 55, Cost 19986.974877\n",
      "Test 56, Cost 20020.869631\n",
      "Test 57, Cost 20016.957706\n",
      "Test 58, Cost 20024.287409\n",
      "Test 59, Cost 20075.838080\n",
      "Test 60, Cost 20060.581170\n",
      "Test 61, Cost 19984.800463\n",
      "Test 62, Cost 20058.557961\n",
      "Test 63, Cost 20098.728825\n",
      "Test 64, Cost 20053.942440\n",
      "Test 65, Cost 20083.574989\n",
      "Test 66, Cost 19997.447079\n",
      "Test 67, Cost 19971.667328\n",
      "Test 68, Cost 20016.177605\n",
      "Test 69, Cost 20059.598767\n",
      "Test 70, Cost 20057.996216\n",
      "Test 71, Cost 20063.632718\n",
      "Test 72, Cost 20029.722922\n",
      "Test 73, Cost 20048.510391\n",
      "Test 74, Cost 20139.063202\n",
      "Test 75, Cost 20076.726715\n",
      "Test 76, Cost 20049.816879\n",
      "Test 77, Cost 20108.880889\n",
      "Test 78, Cost 20011.553556\n",
      "Test 79, Cost 20039.303561\n",
      "Test 80, Cost 20048.793282\n",
      "Test 81, Cost 20112.959457\n",
      "Test 82, Cost 20086.065536\n",
      "Test 83, Cost 20070.484565\n",
      "Test 84, Cost 20058.702361\n",
      "Test 85, Cost 20085.371999\n",
      "Test 86, Cost 20095.236005\n",
      "Test 87, Cost 20116.714831\n",
      "Test 88, Cost 20151.795985\n",
      "Test 89, Cost 20065.754055\n",
      "Test 90, Cost 20070.819635\n",
      "Test 91, Cost 20097.512034\n",
      "Test 92, Cost 20102.745124\n",
      "Test 93, Cost 20091.564970\n",
      "Test 94, Cost 20124.876530\n",
      "Test 95, Cost 20169.187171\n",
      "Test 96, Cost 20062.645004\n",
      "Test 97, Cost 20132.430300\n",
      "Test 98, Cost 20057.634827\n",
      "Test 99, Cost 20147.392275\n",
      "Pass 100, Batch 0, Cost 29078.714844\n",
      "Pass 100, Batch 1, Cost 19023.205078\n",
      "Pass 100, Batch 2, Cost 25691.774457\n",
      "Test 100, Cost 20060.919151\n",
      "Test 101, Cost 20131.709969\n",
      "Test 102, Cost 20056.499218\n",
      "Test 103, Cost 20084.716656\n",
      "Test 104, Cost 20081.548593\n",
      "Test 105, Cost 20096.742751\n",
      "Test 106, Cost 20101.453762\n",
      "Test 107, Cost 20089.491364\n",
      "Test 108, Cost 20174.685706\n",
      "Test 109, Cost 20120.953016\n",
      "Test 110, Cost 20078.660555\n",
      "Test 111, Cost 20107.490863\n",
      "Test 112, Cost 20151.445594\n",
      "Test 113, Cost 20123.325230\n",
      "Test 114, Cost 20094.494272\n",
      "Test 115, Cost 20102.559268\n",
      "Test 116, Cost 20144.542099\n",
      "Test 117, Cost 20120.871174\n",
      "Test 118, Cost 20114.841782\n",
      "Test 119, Cost 20072.191483\n",
      "Test 120, Cost 20074.592435\n",
      "Test 121, Cost 20142.405380\n",
      "Test 122, Cost 20213.436074\n",
      "Test 123, Cost 20090.904413\n",
      "Test 124, Cost 20074.109976\n",
      "Test 125, Cost 20078.749935\n",
      "Test 126, Cost 20144.650220\n",
      "Test 127, Cost 20027.028694\n",
      "Test 128, Cost 20159.249141\n",
      "Test 129, Cost 20185.059366\n",
      "Test 130, Cost 20080.257707\n",
      "Test 131, Cost 20062.711689\n",
      "Test 132, Cost 20051.744383\n",
      "Test 133, Cost 20121.980048\n",
      "Test 134, Cost 20090.948577\n",
      "Test 135, Cost 20092.979942\n",
      "Test 136, Cost 20087.089652\n",
      "Test 137, Cost 20096.284526\n",
      "Test 138, Cost 20105.694831\n",
      "Test 139, Cost 20166.768521\n",
      "Test 140, Cost 20140.645176\n",
      "Test 141, Cost 20108.059758\n",
      "Test 142, Cost 20115.756768\n",
      "Test 143, Cost 20106.585636\n",
      "Test 144, Cost 20101.927626\n",
      "Test 145, Cost 20121.331097\n",
      "Test 146, Cost 20114.270449\n",
      "Test 147, Cost 20099.822642\n",
      "Test 148, Cost 20039.502978\n",
      "Test 149, Cost 20092.477655\n",
      "Test 150, Cost 20078.379035\n",
      "Test 151, Cost 20178.992313\n",
      "Test 152, Cost 20106.709427\n",
      "Test 153, Cost 20148.640053\n",
      "Test 154, Cost 20104.799317\n",
      "Test 155, Cost 20087.005355\n",
      "Test 156, Cost 20106.755953\n",
      "Test 157, Cost 20100.549044\n",
      "Test 158, Cost 20203.183084\n",
      "Test 159, Cost 20122.694827\n",
      "Test 160, Cost 20085.475705\n",
      "Test 161, Cost 20089.693351\n",
      "Test 162, Cost 20077.071864\n",
      "Test 163, Cost 20073.728961\n",
      "Test 164, Cost 20109.239807\n",
      "Test 165, Cost 20143.868712\n",
      "Test 166, Cost 20107.467504\n",
      "Test 167, Cost 20080.072661\n",
      "Test 168, Cost 20172.814121\n",
      "Test 169, Cost 20071.979836\n",
      "Test 170, Cost 20112.914684\n",
      "Test 171, Cost 20104.061585\n",
      "Test 172, Cost 20100.221676\n",
      "Test 173, Cost 20097.412335\n",
      "Test 174, Cost 20080.398056\n",
      "Test 175, Cost 20119.833298\n",
      "Test 176, Cost 20080.848022\n",
      "Test 177, Cost 20117.051461\n",
      "Test 178, Cost 20129.869421\n",
      "Test 179, Cost 20114.368360\n",
      "Test 180, Cost 20077.900059\n",
      "Test 181, Cost 20100.151315\n",
      "Test 182, Cost 20158.916091\n",
      "Test 183, Cost 20184.530637\n",
      "Test 184, Cost 20145.552619\n",
      "Test 185, Cost 20137.226536\n",
      "Test 186, Cost 20080.121342\n",
      "Test 187, Cost 20130.226890\n",
      "Test 188, Cost 20114.132394\n",
      "Test 189, Cost 20098.741143\n",
      "Test 190, Cost 20130.050482\n",
      "Test 191, Cost 20115.298571\n",
      "Test 192, Cost 20108.893720\n",
      "Test 193, Cost 20059.978132\n",
      "Test 194, Cost 20099.288589\n",
      "Test 195, Cost 20025.596357\n",
      "Test 196, Cost 20111.834821\n",
      "Test 197, Cost 20119.872657\n",
      "Test 198, Cost 20119.081675\n",
      "Test 199, Cost 20072.560510\n",
      "Pass 200, Batch 0, Cost 26270.830078\n",
      "Pass 200, Batch 1, Cost 22019.523438\n",
      "Pass 200, Batch 2, Cost 25527.448370\n",
      "Test 200, Cost 20204.402842\n",
      "Test 201, Cost 20087.393581\n",
      "Test 202, Cost 20120.807878\n",
      "Test 203, Cost 20103.769987\n",
      "Test 204, Cost 20191.215324\n",
      "Test 205, Cost 20182.653445\n",
      "Test 206, Cost 20192.687646\n",
      "Test 207, Cost 20147.275621\n",
      "Test 208, Cost 20131.391261\n",
      "Test 209, Cost 20172.906958\n",
      "Test 210, Cost 20137.669757\n",
      "Test 211, Cost 20115.703720\n",
      "Test 212, Cost 20073.252463\n",
      "Test 213, Cost 20120.975068\n",
      "Test 214, Cost 20125.619410\n",
      "Test 215, Cost 20094.421969\n",
      "Test 216, Cost 20179.421849\n",
      "Test 217, Cost 20114.859105\n",
      "Test 218, Cost 20133.612826\n",
      "Test 219, Cost 20069.560471\n",
      "Test 220, Cost 20078.705060\n",
      "Test 221, Cost 20080.139749\n",
      "Test 222, Cost 20141.061678\n",
      "Test 223, Cost 20120.218971\n",
      "Test 224, Cost 20146.462747\n",
      "Test 225, Cost 20184.391527\n",
      "Test 226, Cost 20088.880715\n",
      "Test 227, Cost 20106.059308\n",
      "Test 228, Cost 20110.612886\n",
      "Test 229, Cost 20102.162815\n",
      "Test 230, Cost 20110.962340\n",
      "Test 231, Cost 20061.220256\n",
      "Test 232, Cost 20053.715559\n",
      "Test 233, Cost 20061.986922\n",
      "Test 234, Cost 20092.874915\n",
      "Test 235, Cost 20103.380608\n",
      "Test 236, Cost 20124.687004\n",
      "Test 237, Cost 20090.101007\n",
      "Test 238, Cost 20107.401149\n",
      "Test 239, Cost 20127.207591\n",
      "Test 240, Cost 20112.447251\n",
      "Test 241, Cost 20129.109773\n",
      "Test 242, Cost 20196.895673\n",
      "Test 243, Cost 20230.020985\n",
      "Test 244, Cost 20132.560708\n",
      "Test 245, Cost 20084.699371\n",
      "Test 246, Cost 20162.730003\n",
      "Test 247, Cost 20068.395678\n",
      "Test 248, Cost 20062.961737\n",
      "Test 249, Cost 20153.350064\n",
      "Test 250, Cost 20089.524436\n",
      "Test 251, Cost 20063.361636\n",
      "Test 252, Cost 20080.304607\n",
      "Test 253, Cost 20103.099685\n",
      "Test 254, Cost 20102.951036\n",
      "Test 255, Cost 20128.331451\n",
      "Test 256, Cost 20077.935468\n",
      "Test 257, Cost 20118.023049\n",
      "Test 258, Cost 20123.488924\n",
      "Test 259, Cost 20144.867232\n",
      "Test 260, Cost 20069.428145\n",
      "Test 261, Cost 20056.974057\n",
      "Test 262, Cost 20144.425914\n",
      "Test 263, Cost 20138.675250\n",
      "Test 264, Cost 20111.595171\n",
      "Test 265, Cost 20137.630394\n",
      "Test 266, Cost 20036.899670\n",
      "Test 267, Cost 20168.614940\n",
      "Test 268, Cost 20069.780292\n",
      "Test 269, Cost 20160.588387\n",
      "Test 270, Cost 20121.647385\n",
      "Test 271, Cost 20078.818786\n",
      "Test 272, Cost 20053.685390\n",
      "Test 273, Cost 20092.695715\n",
      "Test 274, Cost 20198.219063\n",
      "Test 275, Cost 20159.019170\n",
      "Test 276, Cost 20077.246953\n",
      "Test 277, Cost 20200.212769\n",
      "Test 278, Cost 20101.063794\n",
      "Test 279, Cost 20099.151128\n",
      "Test 280, Cost 20147.073465\n",
      "Test 281, Cost 20111.771125\n",
      "Test 282, Cost 20149.601737\n",
      "Test 283, Cost 20094.366137\n",
      "Test 284, Cost 20154.444491\n",
      "Test 285, Cost 20177.140237\n",
      "Test 286, Cost 20119.902465\n",
      "Test 287, Cost 20057.071897\n",
      "Test 288, Cost 20038.866929\n",
      "Test 289, Cost 20142.053524\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test 290, Cost 20107.495350\n",
      "Test 291, Cost 20136.340019\n",
      "Test 292, Cost 20104.667335\n",
      "Test 293, Cost 20181.995960\n",
      "Test 294, Cost 20157.259602\n",
      "Test 295, Cost 20132.599551\n",
      "Test 296, Cost 20233.785754\n",
      "Test 297, Cost 20090.803309\n",
      "Test 298, Cost 20132.596757\n",
      "Test 299, Cost 20055.401674\n"
     ]
    }
   ],
   "source": [
    "# 训练\n",
    "trainer.train(\n",
    "    reader=paddle.batch(\n",
    "        paddle.reader.shuffle(train(), buf_size=500),\n",
    "        batch_size=256),\n",
    "    feeding=feeding,\n",
    "    event_handler=event_handler,\n",
    "    num_passes=300)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "训练完成后我们可将结果参数打印出来："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Result Parameters as below:\n",
      "(1137.4955, 608.8786)\n",
      "a =  7.11512781259\n",
      "b =  -64.529399425\n"
     ]
    }
   ],
   "source": [
    "print(\"Result Parameters as below:\")\n",
    "a = parameters.get('___fc_layer_0__.w0')[0]\n",
    "b = parameters.get('___fc_layer_0__.wbias')[0]\n",
    "print(a, b)\n",
    "\n",
    "x0 = X_RAW[0]\n",
    "y0 = a * CODEMASTER_TRAIN_DATA[0][0] + b\n",
    "\n",
    "x1 = X_RAW[1]\n",
    "y1 = a * CODEMASTER_TRAIN_DATA[1][0] + b\n",
    "\n",
    "a = (y0 - y1) / (x0 - x1)\n",
    "b = (y1 - a * x1)\n",
    "\n",
    "print 'a = ', a\n",
    "print 'b = ', b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5 - 学习曲线\n",
    "\n",
    "输出成本的变化情况，也就是学习曲线，对模型进行分析，使用matplotlib展示学习曲线："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZsAAAEWCAYAAACwtjr+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VPW9//HXJxskYQmQBGUTFAiV1jUiCmqo1WqvFrXa\n6q27rbYuXfz96tX+7r12ud5rV697r4q71VKtitZbayuoIChBUQsiRFRWJewkAbJ9fn+cb2CISQiS\nyZlM3s/HYx4z53vOnPOZCcx7zjnf+R5zd0RERJIpI+4CREQk/SlsREQk6RQ2IiKSdAobERFJOoWN\niIgkncJGRESSTmEj0gHM7Bgzey+mbS8ws7IOWtc3zeyvCdNuZiM7Yt1hfVVmtn9HrU+6DoWNdDoz\n+9DMvtSs7UIzmxlXTc2FehrCh+NmM5tvZqe0try7v+LuJR1cw/DwYV8Vbp+Y2bNmdkKzbY919xnt\nXFdWW8u5+yPufmIHlI+ZzTCzbzVbfy93X9oR65euRWEj0rrZ7t4LKACmAFPNrF/zhXb3Ad4BCkId\nBwMvAE+a2YUdvZFOeB3SjSlsJCWZ2efCN+ON4TDRVxPm7fKNOXGvyCI3mdmasEfyjpl9PszrYWa/\nNrNlYS/hd2aWu7ta3L0RuBfIBQ4wszIzW2Fm/2JmHwP3NbUl1DTUzP5kZpVmts7MbkuYd7GZvWtm\nG8zseTPbrz3vibt/7O43Az8BfmFmGWF9O/YUzWycmZWH1/6Jmf02PP3lcL8x7CUdFd63WeH9Wgf8\npJU9zK+Y2VIzW2tmv0rY7k/M7OGE17Vj78nMbgCOAW4L27stLLPjsJyZ9TWzB8N79JGZ/WvCui80\ns5nh77XBzD4ws5MTtnVhqGlLmPfN9ryHEh+FjaQcM8sGngH+ChQDVwGPmFl7DlOdCBwLjAb6Al8H\n1oV5N4b2Q4CRwGDg39tRTxbwLaAKWBKa9wH6A/sBlzZbPhN4FvgIGB6281iYNxn4MXAGUAS8Ajza\njteV6E9E70tL78fNwM3u3gc4AJga2o8N9wXhUNbsMH0ksBQYCNzQyvZOB0qBw4DJwMW7K9Dd/x/R\na7sybO/KFha7lehvtD9wHHA+cFHC/COB94BC4JfAlPBlIh+4BTjZ3XsDRwPzd1eTxEthI3F5Kuy1\nbDSzjcAdCfPGA72AG9291t1fJPrwPqcd660DegNjAHP3d919tZkZUSj80N3Xu/sW4D+Bs9tY1/hQ\n28dh26e7+6YwrxG43t23u/vWZs8bBwwCfuTu1e6+zd2b9ha+A/xXqKs+1HBIe/duglXhvn8rr3+k\nmRW6e5W7z9ndutz9Vnevb+F1NPlFeM+WAf9N+/4ObQqBfDZwnbtvcfcPgd8A5yUs9pG73+3uDcAD\nwL5EoQjR+/95M8t199XuvmBva5LkUthIXE5z94KmG3B5wrxBwPJw+KrJR0R7CG0KwXQbcDuwxszu\nMrM+RHsRecC8hID7S2hvzZxQX6G7j3f3vyXMq3T3ba08byjRB2V9C/P2A25OqGE9YO15bQmall3f\nwrxLiPbeFpnZXGujU0OwvB3bS1zmI6K/z94qBLLD+hLXnfg+fNz0wN1rwsNe7l4NfIMouFeb2Z/N\nbEwH1CRJpLCRVLQKGNp0/D4YBqwMj6uJgqPJPolPdvdb3P1w4ECiD94fAWuBrcDYhJDrG068fxZt\nDZe+HBjWygn35cBliUHr7rnu/uoebPt0YA3RIaZdi3Jf4u7nEB1m+wXweDjs1Fq97Rn2fWjC42Hs\n3LNq8++wm3WvJdoLS9yjS/wbt8ndn3f3E4j2dhYBd7fneRIfhY2koteAGuAaM8u26DckpxLOexAd\nnz/DzPLCyeZLmp5oZkeY2ZHhvE81sA1oDHtJdwM3mVlxWHawmX05CfW/DqwGbjSzfDPraWYTwrzf\nAdeZ2dhQQ18zO6s9KzWzgWZ2JXA90eGnxhaWOdfMisK8jaG5EagM95/lNy4/MrN+ZjYU+D7wh9A+\nHzjWzIaZWV/gumbP+6S17YVDY1OBG8ysdziMeDXwcEvLJwrvw+QQotuJzqV96r2Q1KKwkZTj7rVE\n4XIy0TfgO4Dz3X1RWOQmoJbow+wB4JGEp/chCpUNRIdl1gG/CvP+BagA5pjZZuBvtHySfW/rbwj1\njwSWASuIDvvg7k8S7XE8Fmr4R3idbdloZtXAO8BXgLPc/d5Wlj0JWGBmVUSdBc52963hMNQNwKxw\nCG/8Hrykp4F5ROHyZ6Ju4Lj7C0TB83aY/2yz590MnBl6k93SwnqvIvpCsBSYCfyeqNff7mQQBdMq\nokOJxwHf3YPXIzEwXTxNRESSTXs2IiKSdAobERFJOoWNiIgkncJGRESSTgPvBYWFhT58+PC4yxAR\n6VLmzZu31t3b+nE0oLDZYfjw4ZSXl8ddhohIl2JmH+1+KR1GExGRTqCwERGRpFPYiIhI0ilsREQk\n6RQ2IiKSdAobERFJOoWNiIgkncJmL017axUPz2lXN3MRkW5LYbOXnv/Hx9z64hJ0qQYRkdYpbPbS\ncSVFfLJ5O++u3hJ3KSIiKUths5fKRkdDAs1YvCbmSkREUpfCZi8V9+nJ2EF9mLGoMu5SRERSlsKm\nA5SVFDFv2QY2ba2LuxQRkZSksOkAk0qKaWh0Zi5ZG3cpIiIpSWHTAQ4ZWkCfnlnMeE/nbUREWqKw\n6QBZmRkcM7qIGYsr1QVaRKQFCpsOMqmkmMot21mwanPcpYiIpByFTQc5LnSBfmmxeqWJiDSnsOkg\nRb178IXBfZm+SOdtRESaU9h0oLKSIt5YtoFNNeoCLSKSSGHTgcpKimh0eKVCh9JERBIpbDrQIUP7\nUZCXzXSNJiAisguFTQfKzDCOGVXES4sraWxUF2gRkSYKmw42qaSItVXbWbhaXaBFRJoobDrYsaEL\ntHqliYjspLDpYIW9enDQkL7M0O9tRER2UNgkQVlJMW8u28DGmtq4SxERSQkKmyRo6gL9skaBFhEB\nFDZJcfCQAvrlZWsUaBGRIOlhY2aZZvammT0bpkeY2WtmVmFmfzCznNDeI0xXhPnDE9ZxXWh/z8y+\nnNB+UmirMLNrE9pb3EZnycwwjh1dxEvvqQu0iAh0zp7N94F3E6Z/Adzk7iOBDcAlof0SYENovyks\nh5kdCJwNjAVOAu4IAZYJ3A6cDBwInBOWbWsbnaaspIh11bX8Y9Wmzt60iEjKSWrYmNkQ4J+Ae8K0\nAV8EHg+LPACcFh5PDtOE+ceH5ScDj7n7dnf/AKgAxoVbhbsvdfda4DFg8m620WmOHVWEGRpNQESE\n5O/Z/DdwDdAYpgcAG929PkyvAAaHx4OB5QBh/qaw/I72Zs9prb2tbezCzC41s3IzK6+s7NhQGNCr\nBwcNKWDGYp23ERFJWtiY2SnAGnefl6xt7C13v8vdS929tKioqMPXXza6iPnLN7K+Wl2gRaR7S+ae\nzQTgq2b2IdEhri8CNwMFZpYVlhkCrAyPVwJDAcL8vsC6xPZmz2mtfV0b2+hUk8YU4w6vLNGhNBHp\n3pIWNu5+nbsPcffhRCf4X3T3bwLTgTPDYhcAT4fH08I0Yf6L7u6h/ezQW20EMAp4HZgLjAo9z3LC\nNqaF57S2jU510OC+9M/PYcZ7ChsR6d7i+J3NvwBXm1kF0fmVKaF9CjAgtF8NXAvg7guAqcBC4C/A\nFe7eEM7JXAk8T9TbbWpYtq1tdKqMDOO40RoFWkTEoh0BKS0t9fLy8g5f79PzV/L9x+bz1BUTOGRo\nQYevX0QkTmY2z91Ld7ecRhBIsmNCF2iNJiAi3ZnCJsn65+dwyNACpuu8jYh0YwqbTlA2upi3V2xk\nXdX2uEsREYmFwqYTTBpTFLpAaxRoEemeFDad4POD+lLYK4fpOm8jIt2UwqYTZGQYx44q4uXFlTSo\nC7SIdEMKm05SNqaYDTV1vLViY9yliIh0OoVNJzl2VCEZhkYTEJFuSWHTSQryoi7QL+m8jYh0Qwqb\nTjSppJi3VmxirbpAi0g3o7DpRGUlxQC8vFiH0kSke1HYdKKxg/pQ2KuHRhMQkW5HYdOJmkaBfmWJ\nukCLSPeisOlkZSVFbKypY/5ydYEWke5DYdPJjh1VFLpAq1eaiHQfCptO1jcvm8OG9dPvbUSkW1HY\nxGDSmGLeWbmJNVu2xV2KiEinUNjE4LjRRQC8vFijQItI96CwicHYQX0o6t1D521EpNtQ2MTAzCgb\nHY0CXd/QGHc5IiJJp7CJSVlJMZu31asLtIh0CwqbmEwcVUhmhqlXmoh0CwqbmPTNzebwYf109U4R\n6RYUNjE6rqSIBas2s2azukCLSHpT2MRoUhgFeoZGgRaRNKewidHn9u3NwD49eEnnbUQkzSlsYmQW\njQL98hJ1gRaR9KawidmkkmK2bKvnjWXqAi0i6UthE7MJowrJyjCNJiAiaU1hE7M+PbM5fL9+unqn\niKQ1hU0KKCsp5t3Vm/lEXaBFJE0pbFJAWUk0CrR6pYlIulLYpIAx+/Rmnz49NZqAiKQthU0KMDPK\nSoqYuWQtdeoCLSJpSGGTIspKitmyvZ55H22IuxQRkQ6nsEkRE0YOCF2gdd5GRNJP0sLGzHqa2etm\n9paZLTCzn4b2EWb2mplVmNkfzCwntPcI0xVh/vCEdV0X2t8zsy8ntJ8U2irM7NqE9ha3kcp698ym\ndHg//d5GRNJSMvdstgNfdPeDgUOAk8xsPPAL4CZ3HwlsAC4Jy18CbAjtN4XlMLMDgbOBscBJwB1m\nlmlmmcDtwMnAgcA5YVna2EZKm1RSzKKPt7B609a4SxER6VBJCxuPVIXJ7HBz4IvA46H9AeC08Hhy\nmCbMP97MLLQ/5u7b3f0DoAIYF24V7r7U3WuBx4DJ4TmtbSOllYVRoNUFWkTSTVLP2YQ9kPnAGuAF\n4H1go7vXh0VWAIPD48HAcoAwfxMwILG92XNaax/Qxjaa13epmZWbWXllZfwf8KMH9mJQ3546byMi\naSepYePuDe5+CDCEaE9kTDK3t6fc/S53L3X30qKiorjLiUaBLilmZsVaauvVBVpE0ken9EZz943A\ndOAooMDMssKsIcDK8HglMBQgzO8LrEtsb/ac1trXtbGNlFdWUkSVukCLSJpJZm+0IjMrCI9zgROA\nd4lC58yw2AXA0+HxtDBNmP+iu3toPzv0VhsBjAJeB+YCo0LPsxyiTgTTwnNa20bKmzCykOxMjQIt\nIuklmXs2+wLTzextomB4wd2fBf4FuNrMKojOr0wJy08BBoT2q4FrAdx9ATAVWAj8BbgiHJ6rB64E\nnicKsalhWdrYRsrr1SOLI4b313kbEUkrFu0ISGlpqZeXl8ddBgB3vfw+//ncIl699osMKsiNuxwR\nkVaZ2Tx3L93dchpBIAVNCl2gtXcjIulCYZOCRhb3YnBBrs7biEjaUNikoKZRoGepC7SIpAmFTYoq\nKymmuraB8g/Xx12KiMheU9ikqKMPGEBOZgYzFuu8jYh0fQqbFJXfI4txI/ozfZHO24hI16ewSWFl\nJUUsWVPFig01cZciIrJXFDYprExdoEUkTShsUtgBRfkM6ZersBGRLk9hk8KaukC/+v5attc3xF2O\niMhnprBJcZNKiqmpbWDuBxoFWkS6LoVNijuqqQu0RhMQkS5MYZPi8nKyOHL//vq9jYh0aQqbLqCs\npJiKNVUsX68u0CLSNSlsuoCykuiS1dq7EZGuSmHTBexfmM+w/nnM0GgCItJFtStszOys9rRJcuzs\nAr2ObXXqAi0iXU9792yua2ebJElZSRFb6xqYq1GgRaQLymprppmdDHwFGGxmtyTM6gPUJ7Mw2dVR\n+xeSk5XB9EWVHDOqKO5yRET2yO72bFYB5cA2YF7CbRrw5eSWJolyczIZv/8AZizWeRsR6Xra3LNx\n97eAt8zs9+5eB2Bm/YCh7q6ftHeySSVF/PSZhSxbV8OwAXlxlyMi0m7tPWfzgpn1MbP+wBvA3WZ2\nUxLrkhbsGAVaezci0sW0N2z6uvtm4AzgQXc/Ejg+eWVJS0YU5rPfgDyNAi0iXU57wybLzPYFvg48\nm8R6ZDcmlRTz6vtr1QVaRLqU9obNz4Dngffdfa6Z7Q8sSV5Z0prjSorYVtfIax+oC7SIdB3tCht3\n/6O7H+Tu3w3TS939a8ktTVpy1P4D6JGVwXSNJiAiXUh7RxAYYmZPmtmacHvCzIYkuzj5tJ7ZmRx1\nwABe0jhpItKFtPcw2n1Ev60ZFG7PhDaJQdnoIj5YW82Ha6vjLkVEpF3aGzZF7n6fu9eH2/2AfsYe\nkx1doHVBNRHpItobNuvM7Fwzywy3c4F1ySxMWje8MJ8Rhfm65ICIdBntDZuLibo9fwysBs4ELkxS\nTdIOZSVFzNYo0CLSRexJ1+cL3L3I3YuJwuenyStLdqespJjt9Y3MXqodTBFJfe0Nm4MSx0Jz9/XA\nockpSdrjyBH96ZmdwUsaTUBEuoD2hk1GGIATgDBGWpuDeEpy9czO5OgDCpmuTgIi0gW0NzB+A8w2\nsz+G6bOAG5JTkrRXWUkRLy5awwdrqxlRmB93OSIirWrvCAIPEg3C+Um4neHuDyWzMNm9stHqAi0i\nXUO7D4W5+0JgYRJrkT00bEAe+xflM/29Si6aMCLuckREWtXeczZ7zMyGmtl0M1toZgvM7Puhvb+Z\nvWBmS8J9v9BuZnaLmVWY2dtmdljCui4Iyy8xswsS2g83s3fCc24xM2trG+mobHQxc5auY2utukCL\nSOpKWtgA9cD/cfcDgfHAFWZ2IHAt8Hd3HwX8PUwDnAyMCrdLgTthR2eE64EjgXHA9QnhcSfw7YTn\nnRTaW9tG2pk0poja+kZmL10bdykiIq1KWti4+2p3fyM83gK8CwwGJgMPhMUeAE4LjycTXZjN3X0O\nUBCuofNl4AV3Xx+6X78AnBTm9XH3Oe7uwIPN1tXSNtLOuBH9yc3O1AXVRCSlJXPPZgczG070u5zX\ngIHuvjrM+hgYGB4PBpYnPG1FaGurfUUL7bSxjeZ1XWpm5WZWXlnZNT+se2RlcvQBA5jxXiVR5oqI\npJ6kh42Z9QKeAH4QLi29Q9gjSeonZFvbcPe73L3U3UuLirruuKJlY4pZtr6GpRoFWkRSVFLDxsyy\niYLmEXf/U2j+JBwCI9w39dtdCQxNePqQ0NZW+5AW2tvaRloqGx0FpQ6liUiqSmZvNAOmAO+6+28T\nZk0DmnqUXQA8ndB+fuiVNh7YFA6FPQ+caGb9QseAE4Hnw7zNZjY+bOv8ZutqaRtpaWj/PEYW99Lv\nbUQkZSVzyJkJwHnAO2Y2P7T9GLgRmGpmlwAfEY0mDfAc8BWgAqgBLoJoHDYz+zkwNyz3szA2G8Dl\nwP1ALvC/4UYb20hbZaOLeHD2R9TU1pOXo5GERCS1mE4qR0pLS728vDzuMj6zmUvWcu6U15hyQSnH\nf67F/hAiIh3OzOa5e+nuluuU3miSfEeM6EdeTqYG5hSRlKSwSRNRF+hCdYEWkZSksEkjk8YUsWLD\nVt6vVBdoEUktCps0UlaiUaBFJDUpbNLI4IJcRhX30u9tRCTlKGzSzKQxxbz+wXqqt9fHXYqIyA4K\nmzRTNrqI2oZGXn1/XdyliIjsoLBJM6XD+5Ofk6nzNiKSUhQ2aSYnK4MJI9UFWkRSi8ImDZWVFLNy\n41Yq1lTFXYqICKCwSUtlJdEo0BpNQERShcImDQ0qyKVkYG+enr+KKvVKE5EUoLBJU98tO4BFH29h\n8m0zeb9Sh9NEJF4KmzR12qGDeeiScWyoqWPybbP464KP4y5JRLoxhU0aO/qAQp65aiL7F+Vz6UPz\n+M1f36OhUT3URKTzKWzS3OCCXKZedhRnHT6EW1+s4JIH5rKppi7uskSkm1HYdAM9szP55ZkH8fPT\nPs+sirV89faZLPp4c9xliUg3orDpJsyM88bvx2OXjmdrbQOn3/4q095aFXdZItJNKGy6mcP368+z\nV01k7KA+fO/RN7nhzwupb2iMuywRSXMKm26ouE9Pfv/t8Zx/1H7c/coHnDflddZVbY+7LBFJYwqb\nbionK4OfTf48vz7rYOYt28Cpt87k7RUb4y5LRNKUwqabO/PwITzxnaMxM8783Wymli+PuyQRSUMK\nG+ELQ/oy7coJlO7Xj2sef5t/feodaut1HkdEOo7CRgAY0KsHD148jkuP3Z+H5yzjnLvnsGbztrjL\nEpE0obCRHbIyM/jxVz7HreccysJVmznl1pnM+2h93GWJSBpQ2MinnHrwIJ66YgK5OZmcfdccHpr9\noS7EJiJ7RWEjLSrZpzfTrpzIxJGF/NvTC/jR42+zra4h7rJEpItS2Eir+uZmM+WCI/je8aN4fN4K\nzvrdbFZu3Bp3WSLSBSlspE0ZGcbVJ4zmrvMO54O11Zx660xerVgbd1ki0sUobKRdThy7D09fOYH+\n+TmcO+U17n55qc7jiEi7KWyk3Q4o6sVTV0zgxAP34Ybn3uV7j82nplaXnRaR3VPYyB7p1SOLO889\njGtOKuHZt1dxxh2v8tG66rjLEpEUp7CRPWZmXF42kgcuGsfqTds49daZTH9vTdxliUgKU9jIZ3bs\n6CKevWoig/vlcfH9c7nl70to1GWnRaQFChvZK0P75/Gn7x7N5IMH8dsXFnPZw/PYvE2XnRaRXSls\nZK/l5mRy0zcO4d9POZAXF63htNtmUbFmS9xliUgKUdhIhzAzLp44gke+dSSbt9Ux+bZZ/OUfq+Mu\nS0RSRNLCxszuNbM1ZvaPhLb+ZvaCmS0J9/1Cu5nZLWZWYWZvm9lhCc+5ICy/xMwuSGg/3MzeCc+5\nxcysrW1I5xi//wCeuWoiIwf25jsPv8Ev/7KIBp3HEen2krlncz9wUrO2a4G/u/so4O9hGuBkYFS4\nXQrcCVFwANcDRwLjgOsTwuNO4NsJzztpN9uQTrJv31ymXjaes48Yyh0z3uei++eysaY27rJEJEZJ\nCxt3fxloPj79ZOCB8PgB4LSE9gc9MgcoMLN9gS8DL7j7enffALwAnBTm9XH3OR79jP3BZutqaRvS\niXpkZXLj1w7iv874AnPeX8ept81k4arNcZclIjHp7HM2A9296UD+x8DA8HgwkHg94hWhra32FS20\nt7WNTzGzS82s3MzKKysrP8PLkd05Z9wwHrtsPHX1zhl3zuLp+SvjLklEYhBbB4GwR5LUg/m724a7\n3+Xupe5eWlRUlMxSurXDhvXjmasmctDgAr7/2Hx+9sxC6hp02WmR7qSzw+aTcAiMcN/0s/OVwNCE\n5YaEtrbah7TQ3tY2JEZFvXvwyLeP5MKjh3PvrA84957XWFu1Pe6yRKSTdHbYTAOaepRdADyd0H5+\n6JU2HtgUDoU9D5xoZv1Cx4ATgefDvM1mNj70Qju/2bpa2obELDszg598dSy//frBzF++kVNvnclb\nyzfGXZaIdIJkdn1+FJgNlJjZCjO7BLgROMHMlgBfCtMAzwFLgQrgbuByAHdfD/wcmBtuPwtthGXu\nCc95H/jf0N7aNiRFnHHYEJ747tFkmHHW/8xm6tzlu3+SiHRppmuSREpLS728vDzuMrqV9dW1fO/R\nN5lZsZZvHjmM608dS06Wfmcs0pWY2Tx3L93dcvqfLbHpn5/D/RcdwWXH7c8jry3jnLvn8MnmbXGX\nJSJJoLCRWGVlZnDdyZ/jtn8+lHdXb+aUW2dS/mHzn2eJSFensJGUcMpBg3jy8gnk5WRy9l1zeGj2\nh7rstEgaUdhIyijZpzfTrpzIMaMK+benF3DN42+zra4h7rJEpAMobCSl9M3NZsoFR/C940fxx3kr\n+Pr/zGblxq1xlyUie0lhIyknI8O4+oTR3HXe4SytrObUW2fy6vtr4y5LRPaCwkZS1olj9+GpKybQ\nLy+b86a8zj2vLNV5HJEuSmEjKW1kcS+eumICX/pcMf/x53f5wR/ms7VW53FEuhqFjaS83j2zufOb\nh/OjL5cw7a1VnH7HLJatq4m7LBHZAwob6RIyMowrJo3kvguPYNXGrZx620xeWqzLQoh0FQob6VLK\nSop55qqJ7Nu3Jxfe9zq3T6/QeRyRLkBhI13OfgPy+dPlR3PKQYP41fPvcfkjb1C1vT7uskSkDQob\n6ZLycrK45exD+Nd/+hx/XfgJp98+i6WVVXGXJSKtUNhIl2VmfOuY/XnoknGsq65l8m2z+NvCT+Iu\nS0RaoLCRLu/oAwp55qqJDC/M51sPlvPbFxbT2KjzOCKpRGEjaWFwQS5//M5RnHn4EG75+xK+/WA5\nm7bWxV2WiAQKG0kbPbMz+dWZB/HzyWN5aXElp90+i8WfbIm7LBFBYSNpxsw476jhPHrpeKq213Pa\n7bN47p3VcZcl0u0pbCQtHTG8P89eNZEx+/Tm8kfe4Mb/XUSDzuOIxEZhI2lrYJ+ePHrpeL555DB+\n99L7XHjf62yoro27LJFuSWEjaa1HViY3nP4FfvG1L/Da0vWcettMFqzaFHdZIt2Owka6hW8cMYyp\n3zmKhkbna3e+ylNvroy7JJFuRWEj3cYhQwt45qqJHDykgB/8YT4/fWYBdQ2NcZcl0i0obKRbKezV\ng4e/dSQXTRjOfbM+5Nx7XmNt1fa4yxJJewob6XayMzO4/tSx3PSNg3lrxUZOvXUm85dvjLsskbSm\nsJFu6/RDh/DEd48mM8P4+u9mc/v0Cl5aXEnFmipdDVSkg2XFXYBInMYO6sszV07ke4+9ya+ef2+X\nef3zcxhckBvd+u16P6RfLn1zszGzmCpPD+5Oo0OjO43uuIMnTDc60Gza3XHYMd3YGJ5HNG1AXo9M\nevXIIjc7U3+jFKGwkW6vX34OD148jtWbtrFy41ZWbtga3YfHFZVVvLS4kq11u+7t5OdkNguhvHDf\nk8EFeRT37kFGRtf6oNte38CG6jrWV9dGt5pa1ldt3/m4eudty7b6hABoCo6d4bEzJKJ7Z9dw6Yxr\n3mUY5PfIolePLPLDrVePTPJzdm3r3TOL/JzMFpbNIj8EV36PLLIzU+NgUH1DIzV1DWyrbaAm3LbW\n1Uf3tQ1srUtor63fMb212fJN0/dcUMqQfnlJrVlhI0I0zM2gglwGFeRyxPBPz3d3NtTUhSCqYcWG\nrbsE05t4iBYjAAALSElEQVTLN7KxZteBP7MzjX37fnrPaEi437dvLjlZyfvwcnc2b6vfJSA2VNey\nrrqWDTW1rKsK96F9fXVtqxehM4OC3Gz65+fQPz+HEYX59O6ZTVaGYRa9fxkGGWYYTdOhLSOxjR3t\nNJs2i9aVkdAO7LqesP7EZZpvv6HRqamtp2p7A9Xb66naXk/19nqqE9rWVdWwZVvUVr29nrqG9iVf\nTlYGvZsHV9PjnE+3RSEWPe6RncG2up0f+ImhsLU2CopdAqRuZ/uuyzZQu4e9KDMzjLzsTHJzwi07\nk7ycTPJyshjQq0en7P0pbETawcx2fNB+YUjfFpep3l7f4p7Ryo1bmblkLZ9s2bbLt3kzKO7dI4RQ\nXkIo9dyxl9Srx87/orX1jWxotnexy62mlvVVOx9vqK6lvpUhenpkZTAgP4f+vXLol5fDiAF59MvP\nYUB+zs77vBwGhPkFeTlkdrG9tD2xvb6B6mbhFN03ULW9bkdI7To/altfXcuy9TVUbWsKtc92vi8v\nhEBuThQEuTlZ5GVnMrBPdtSWvbM9d8fjptDIpGd2FB67tGdn0TMng5zMjNgPJ5qu3x4pLS318vLy\nuMuQNFZb38jHm7axYmPNzkBKCKbVG7d96htr39xs+uRmsbG6ji1tXPq6IC/sdeTl7AjFxFtTgDRN\n5+Xoe2ayNDY6NXWfDq7tdY0hEBLDIQqIntnxh8FnZWbz3L10d8vpX5xIJ8nJymDYgDyGDWj52Hhj\no1NZtX3XENqwlc3b6qK9jMS9joT7gtxsslLkXIJEh/t6hfM9A+MuJoUobERSREaGMbBPTwb26clh\nw/rFXY5Ih9LXIRERSTqFjYiIJJ3CRkREki5tw8bMTjKz98yswsyujbseEZHuLC3DxswygduBk4ED\ngXPM7MB4qxIR6b7SMmyAcUCFuy9191rgMWByzDWJiHRb6Ro2g4HlCdMrQpuIiMQgXcOmXczsUjMr\nN7PyysrKuMsREUlb6fqjzpXA0ITpIaFtF+5+F3AXgJlVmtlHn3F7hcDaz/jcZFJde0Z17RnVtWfS\nta792rNQWo6NZmZZwGLgeKKQmQv8s7svSNL2ytszNlBnU117RnXtGdW1Z7p7XWm5Z+Pu9WZ2JfA8\nkAncm6ygERGR3UvLsAFw9+eA5+KuQ0REunkHgQ50V9wFtEJ17RnVtWdU157p1nWl5TkbERFJLdqz\nERGRpFPYiIhI0ils9lIqDvhpZvea2Roz+0fctSQys6FmNt3MFprZAjP7ftw1AZhZTzN73czeCnX9\nNO6aEplZppm9aWbPxl1LEzP70MzeMbP5ZpYy11M3swIze9zMFpnZu2Z2VArUVBLep6bbZjP7Qdx1\nAZjZD8O/+X+Y2aNm1jNp29I5m88uDPi5GDiBaEicucA57r4w5rqOBaqAB93983HWksjM9gX2dfc3\nzKw3MA84LQXeLwPy3b3KzLKBmcD33X1OnHU1MbOrgVKgj7ufEnc9EIUNUOruKfUjRTN7AHjF3e8x\nsxwgz903xl1Xk/CZsRI40t0/64/IO6qWwUT/1g90961mNhV4zt3vT8b2tGezd1JywE93fxlYH3cd\nzbn7and/IzzeArxLCoxZ55GqMJkdbinxLczMhgD/BNwTdy2pzsz6AscCUwDcvTaVgiY4Hng/7qBJ\nkAXkhh/C5wGrkrUhhc3e0YCfn5GZDQcOBV6Lt5JIOFQ1H1gDvODuKVEX8N/ANUBj3IU048BfzWye\nmV0adzHBCKASuC8cdrzHzPLjLqqZs4FH4y4CwN1XAr8GlgGrgU3u/tdkbU9hI53OzHoBTwA/cPfN\ncdcD4O4N7n4I0Th648ws9sOPZnYKsMbd58VdSwsmuvthRNeMuiIcuo1bFnAYcKe7HwpUAylxHhUg\nHNb7KvDHuGsBMLN+REdiRgCDgHwzOzdZ21PY7J12DfgpO4VzIk8Aj7j7n+Kup7lw2GU6cFLctQAT\ngK+G8yOPAV80s4fjLSkSvhXj7muAJ4kOKcdtBbAiYa/0caLwSRUnA2+4+ydxFxJ8CfjA3SvdvQ74\nE3B0sjamsNk7c4FRZjYifGs5G5gWc00pK5yInwK86+6/jbueJmZWZGYF4XEuUYePRfFWBe5+nbsP\ncffhRP+2XnT3pH3zbC8zyw8dPAiHqU4EYu/56O4fA8vNrCQ0HQ/E2vmkmXNIkUNowTJgvJnlhf+b\nxxOdR02KtB0brTOk6oCfZvYoUAYUmtkK4Hp3nxJvVUD0Tf084J1wfgTgx2EcuzjtCzwQegplAFPd\nPWW6GaeggcCT0ecTWcDv3f0v8Za0w1XAI+HL31LgopjrAXaE8gnAZXHX0sTdXzOzx4E3gHrgTZI4\ndI26PouISNLpMJqIiCSdwkZERJJOYSMiIkmnsBERkaRT2IiISNIpbKRLMbNXw/1wM/vnDl73j1va\nVrKY2Wlm9u9JWnfV7pf6TOst29vRp83sfjM7s435V5rZxXuzDUk9ChvpUty96RfOw4E9Cpsw2GBb\ndgmbhG0lyzXAHXu7kna8rqTr4BruJfq9jKQRhY10KQnf2G8EjgnXB/lhGEjzV2Y218zeNrPLwvJl\nZvaKmU0j/JrczJ4KA0guaBpE0sxuJBr9dr6ZPZK4LYv8Klzz4x0z+0bCumckXD/lkfBLbMzsRouu\n2/O2mf26hdcxGtjeNER/+Lb/OzMrN7PFYVy0pgFC2/W6WtjGDRZdo2eOmQ1M2M6ZCctUJayvtddy\nUmh7Azgj4bk/MbOHzGwW8FAbtZqZ3WbRdZ/+BhQnrONT75O71wAfmlkqDIEjHST2b0Qin9G1wP9t\nur5LCI1N7n6EmfUAZplZ0wi2hwGfd/cPwvTF7r4+DE0z18yecPdrzezKMBhnc2cAhwAHA4XhOS+H\neYcCY4mGZp8FTDCzd4HTgTHu7haGwmlmAtEvtxMNJxpj7ABgupmNBM7fg9eVKB+Y4+7/z8x+CXwb\n+I8WlkvU0mspB+4GvghUAH9o9pwDiQbl3NrG3+BQoCQsO5AoHO81swFtvE/lwDHA67upWboI7dlI\nujgRON+iYXBeAwYAo8K815t9IH/PzN4C5hANpDqKtk0EHg0jQ38CvAQckbDuFe7eCMwnCoxNwDZg\nipmdAdS0sM59iYbDTzTV3RvdfQnRUCtj9vB1JaoFms6tzAt17U5Lr2UM0WCNSzwabqT5QKDT3H1r\neNxarcey8/1bBbwYlm/rfVpDNBKxpAnt2Ui6MOAqd39+l0azMqKh5hOnvwQc5e41ZjYD2JtL4W5P\neNwAZIUx88YRDWx4JnAl0Z5Boq1A32ZtzceOctr5ulpQ5zvHompg5//1esKXTDPLAHLaei1trL9J\nYg2t1fqVlp64m/epJ9F7JGlCezbSVW0BeidMPw9816JLGGBmo63lC2f1BTaEoBkDjE+YV9f0/GZe\nAb4RzkkUEX1Tb/XwjkXX6+kbBhj9IdHht+beBUY2azvLzDLM7ABgf+C9PXhd7fUhcHh4/FWiq5K2\nZREwPNQE0cjFrWmt1pfZ+f7tC0wK89t6n0aTAiNJS8fRno10VW8DDeFw2P3AzUSHfd4IJ7YrgdNa\neN5fgO+E8yrvER1Ka3IX8LaZveHu30xofxI4CniLaG/jGnf/OIRVS3oDT5tZT6Jv+1e3sMzLwG/M\nzBL2QJYRhVgf4Dvuvs3M7mnn62qvu0NtbxG9F23tHRFquBT4s5nVEAVv71YWb63WJ4n2WBaG1zg7\nLN/W+zQB+MmevjhJXRr1WSQmZnYz8Iy7/83M7geedffHYy4rdmZ2KHC1u58Xdy3ScXQYTSQ+/wnk\nxV1ECioE/i3uIqRjac9GRESSTns2IiKSdAobERFJOoWNiIgkncJGRESSTmEjIiJJ9/8BclXl0NmD\nxEYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f204e75ab10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "costs = np.squeeze(costs)\n",
    "plt.plot(costs)\n",
    "plt.ylabel('cost')\n",
    "plt.xlabel('iterations (per hundreds)')\n",
    "plt.title(\"House Price Distributions \")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到图中成本在刚开始收敛较快，随着迭代次数变多，收敛速度变慢，最终收敛到一个较小值。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
